Synthetic Robot Language Development

Cooperating robots can benefit from communication. Our robots create their own adaptable synthetic robot languages (ASRLs). We have shown that robots can develop “basic”, context dependent, and compositional ASRLs using reinforcement learning techniques. (See (Yanco 1994) for a complete description of this work.) We have demonstrated that the robots are able to develop ASRLs using two different reinforcement schemes: task-based reinforcement and individual reinforcement. In task-based reinforcement, the robots only receive positive reinforcement when the task is completed properly. This reinforcement method is preferable in situations where it can not be determined who performed the correct actions to reach the goal, but it is clear that the goal was reached. Individual reinforcement is better suited to tasks where it is clear which of the robots helped to reach the goal. This determination is used to give the robots that helped reach the goal good reinforcement while penalizing the robots that did not contribute toward the group goal. In our model, the robots are able to learn more quickly using the individual reinforcement, but at the expense of convergence. However, most tasks can not easily be decomposed to determine which members of the group acted correctly, and even for those tasks that can be easily decomposed, the overhead necessary to make the determination is often costly. While task-based reinforcement results in longer learning times, it only requires a one-bit decision in allocating reinforcement. The basic ASRL is a simple one-to-one mapping of robot signals to robot actions. The development of this ASRL in simulation and with robots demonstrated that robots could learn to communicate and could adapt their language to changing circumstances. Simulated robots have also created a context dependent ASRL. In a context dependent language, robot words can have different meanings depending on the